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(54) Filtering method to reduce block artifacts in digital video sequences 



(57) The present invention relates generally to the 
domain of digital signal processing and. more precisely, 
the methods and apparatus employed in the reduction 
of visibility of blockiness in images or video sequences 
encoded using block-based coding techniques such as 
MPEG. 

The present invention detects a level of blockiness 
between blocks in an image and applies a variable filter 
across the block boundaries. The filter parameters are 
chosen depending on the level of blockiness detected. If 
no blockiness is detected then no filtering is performed 
on the image. 

An important aspect of the present invention 
reduces the false detection of naturally occurring block- 
iness in images by analysing the image both spatially 
and temporally. 
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Description / 

The present invention relates to the processing of an input signal and more precisely the methods and apparatus 
used for filtering an input signal representing a digital block-based coded image. 

Digital video compression systems are designed to remove irrelevant and less important information from imaae 
sequences in such a way that the best image rendition is achieved for a given data rate. The data rate required to qive 
a faithful reproduction of the original image sequences depends greatiy on the type of picture material in question In 
systems which are designed to compress the image sequence to a constant bit-rate, in particular, it is not possible to 
completely avoid the appearance of compression defects on the decompressed images. Even in systems in which the 
bit-rate .s allowed to vary according to the criticality of video signal it is inevitable that the decompressed images suffer 

y " T °°TT i0n arte,3CtS - The reaSOn for this bein9 that video compression algorithms, which are 
designed to work well for the majority of input material, will, in extreme cases, produce an output data rate which actu- 
ally exceeds the data rate of the original image sequence. Image compression systems are. therefore, designed to qive 
the best picture quality for high percentages of time, accepting that, occasionally, the system might be overloaded and 
is compression artefacts might appear on the output of the system. 

The most common and visually annoying artefact in today's video compression systems is blockiness. Block-based 
image compression systems, such as the commonly used motion-compensated hybrid discrete cosine transform (DCT) 
algorithms, suffer from blockiness as a result of dividing the image into a number of blocks each of which is coded and 
predicted separately. Coding takes place after DCT transformation which provides a method for distinguishing between 
spatial frequency components of greater or lesser importance. Blockiness is an effect in which the boundaries between 
hese coding blocks become visible due to the fact that not enough or too coarsely quantised DCT coefficients are being 
transmitted to g.ve faithful reproduction of the original image. Since the quantisation and the loss of higher-order DCT 
coefficients ,s a consequence of statistical loading of the coder, it is possible, to some extent, to reduce the amount of 
block distribution by applying more sophisticated coding techniques and/or by providing more transmission bandwidth 
25 but it is not possible, in principle, to avoid blockiness altogether. 

Accordingly, the object of the present invention is to reduce the visibility of block distortion in a block-based coded 

image. 

• A H C ^ d ,'?.? to , I"' 54 aSPeCt ° f the preSent invention there is Provided a method of filtering an input signal represent- 
ing a digrtal block-based coded image to produce an output signal, the method comprising the steps of: detecting the 
level of block distortion in the image; and filtering the input signal in dependence on the level of block distortion 

According to a second aspect of the present invention there is provided apparatus for filtering an input signal rep- 
resenting a digital block-based coded image to produce an output filtered signal, the apparatus comprising; a detector 
tortfon 3 distortion; and a filter for filtering the input signal in dependence on the level of block dis- 

The present invention may be applied with advantage to help reduce the visibility of block distortion by applyina 
adaptively varying spatial filters to the output images of compression decoders. This makes it possible to operate com- 
pression systems more efficiently because they no longer have to be designed to cope with statistical worst case 
events, thus wasting valuable bandwidth resource during most of their normal operational time. Instead systems can 
be designed in wh.ch occasional overload is acceptable because the effects are no longer as dramatic as they would 
40 be without the invention. ' wuu,u 

The invention will now be described, by way of example, with reference to the accompanying drawings, in which: 

Figure 1 is a block diagram showing a first embodiment of the present invention; 
Rgure 2 is a block diagram showing a second embodiment of the present invention; 

Figure 3 shows an example of a signal containing severe block distortion and the result after the signal has passed 
through the filtering process of the present invention; and 
Figure 4 is a table which shows a set of possible filter coefficients 

Figure 1 is a block diagram showing a first embodiment of the present invention. Input signal 1 0 is a signal that reo- 
resents a digital block-based coded image. The input signal 10 is processed by the horizontal block detector 11 tern- " 
poral & spatial analyser 13, block-edge frame store 12, vertical block detector 15, temporal & spatial analyser 16 block- 
edge frame store 1 4. horizontal de-blocking filter 1 7. and vertical de-blocking filter 1 8 to reduce the visibility of block dis- 
tortion within the image. The resultant output video signal 19 is then fed to display, storage or other means 

The input signal 10 is first analysed both horizontally and vertically for block distortion. The information gained in 
tne analysis is then used to adaptively control a number of smoothing filters to reduce visibility of block distortion 

The analyse of block distortion makes use of the fact that the block size and location in image compression sys- 
tems .s both known and fixed. Therefore, it is possible to compare spatial differences within each block with those 
across block boundaries. 
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The horizontal analysis is carried out by way of the horizontal block detector 1 1, the temporal & spatial analyser 13 
and the block-edge frame store 12. 

The vertical analysis is carried out by way of the vertical block detector 15, the temporal & spatial analyser 16 and 
the block-edge frame store 14. 
5 The Blockiness Bjj between blocks b t j and £),.•) j is defined as follows: 



Equation 1 



15 where S x y denote the samples of the image and N is the block size 

Equation 1 provides a way of analysing the image to determine the presence of block distortion. It achieves this by 
firstly analysing the image across the block boundary. It then analyses the content of each block on either side of the 
boundary which enables it to detect any discontinuities between the blocks which could amount to block distortion. A 
level of block distortion can then be calculated which represents the a measure of discontinuity between the blocks. 

20 When analysing the spatial detail of the image a compromise needs to be reached regarding the portion of each 
image block that is analysed. If only one pixel either side of the boundary is analysed then insufficient information would 
be available to accurately detect the level of block distortion. On the other hand, if too many pixels either side of the 
boundary are analysed then the level of block distortion detected could be inaccurate. Since block distortion only exists 
between blocks, analysing image data in the centre of a block could lead to unnecessary filtering of the block 

25 In this embodiment, as shown in Equation 1 , the spatial detail is analysed three pixels (or three samples) either side 

of the boundary formed by adjacent blocks. This represents a compromise between evaluating spatial masking across 
the block boundary and allowing for decorrelation of samples which are further away. If the spatial difference across 
block boundaries is significantly greater than that within each of the two adjacent blocks, it is probably due to block dis- 
tortion. Furthermore, if the spatial differences are all similar in magnitude it is likely that any possible block distortion 

30 would be masked by the spatial detail. 

In some instances it may be desirable to increase or decrease the number of pixels analysed between adjacent 
blocks, for example if the size of the blocks was to increase. 

The numerator A in Equation 1 is the sum of the absolute differences across the block boundary, whereas the 
denominator B is the sum of the absolute differences of adjacent samples within the two blocks but near the block edge. 

35 Note that the division A/B in Equation 1 can be replaced by an approximation to the difference log(A) - log(B) for 
the purpose of implementing the algorithm in hardware and that the denominator B has to be checked to be non-zero 
before the division can be performed. For the particular case where B-q = 0 and A > 0 maximum blockiness (Bjj = 10) 
is assumed. 

Both the horizontal and the vertical block detectors 1 1 and 15 perform the detection by way of an implementation 
40 of Equation 1, although it would be appreciated by someone skilled in the art that other detection methods could also 
be used. The operation for the vertical analysis is functionally identical to that of the horizontal analysis apart from the 
fact that the analysis is performed in the vertical direction. 

The input data signal 10 is analysed for horizontal blockiness by the horizontal block detector 11. Occasionally it 
will happen that real image objects coincide exactly with block boundaries. In order to reduce the likelihood of false 
45 detection of block distortion in those cases, the result of the analysis in the previous frame not only of the block in ques- 
tion but, to allow for motion, also of neighbouring blocks are taken into account. If blockiness is detected the data signal 
is passed on to the temporal and spatial analyser 13. In conjunction with the block-edge frame store 12, the temporal 
and spatial analyser 13 is used to reduce the false detection of blockiness by taking into account temporal and spatial 
differences between images. The block-edge frame store 1 2 is used to store block edges so that block edges of different 
so image frames can analysed to provide temporal analysis. This allows motion to be considered in deciding whether 
blockiness actually exists or is naturally occurring within an image. 

For the temporal analysis, blockiness of the current block boundary B i} t is. therefore, calculated as follows: 



. S I^W +y ™" $iN- 1 JN+ y \ 

10 ; — — — , *, . 
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where MIN(A) defines the smallest number in A 
The output from the temporal and spatial analyser 13 is a horizontal control signal 21 which represents the detected 
level of b.ock.ness. It .. then fed into the horizontal de-blocking filter 17 which, depending on the ^alue the horizontal 
control a gnal 20 determined by the horizonta. analysis, applies a variable degree of filtering to reduce M£ "vSEJ 
he distortion to the agnal. Figure 4 shows a set of possible filter coefficients that could be used. Vertical de-blockina 
filter 18 is used to apply the filtering in the vertical direction when vertical blockiness is detected 

The level of smoothing applied to the signal varies according to the amount of blockiness detected - for example 
where severe btockiness is detected across block boundaries a harsh filter, for example Filter 1 from Figure 4 will be 
,„ S^;" h ' r f " " ttle diSt0rti ° n iS detected across b,ock boundaries a much finer filter will be applied for example 
^nTo^Zl~ fe " eSS6ntia,,y n ° b,OCkineSS "° ,iKerin9 ^ ^ ^ R,ter?fr0m Fi9 - Ti: 
theor^oT^ 

It is important to note that the filtering is applied only across those block boundaries where the distortion has been 
is measured, not over any other part of the image. u.sioaion nas Deen 

The actual level of filtering applied to the image is dependant on the level of blockiness detected This allows the 
invention to cope with a wide degree of blockiness. from extreme blockiness down to no blockiness at all. Figure 4 
shows a set of possible f .Iter coefficients that could be used depending on the detected level of blockiness 

Filter 1 (see Figure 4). in particular, has been designed to convert a signal with extreme block distortion (ft,. 10) 
!u ° ^ eCe " W ; S s 6 " n T,r? f ° rm 35 Sh<Wn in R9Ure 3 - The ° ther ,ilters are ^P'^ in ««es of less severe distortion 

bliSSSi ng arte,act3 ' Filter 5 has no filtering 6ffec1 and is used when "° b,ock distortion *™ 

pn J 9Ure 2 f ^ a sec ° nd embodiment of the present invention. Input signal 30. representing a digital block-based 

oHh^XS ' " h /■? ^ 61 10 3 f i,terCOntroller 31 and a de-blocking fNter 35. The filter controHer consols the direSon 
of the filtering and the analysis operation by way of direction control signal 39 

To detect and correct for horizontal block distortion filter controller 31 informs block detector 32. temporal & spatial 

toT e S ler moduli 9 35 ? ^ ^ ° f i^SSiSSS 

5 !2^hiT ? / W £ V ° d ' reCt,0n C0ntrO1 Si9na ' 39 - T ° detect "« correct ,or vertical blockiness filter controller 
3 1 informs block detector 32. temporal & spatial analyser 33 and de-blocking filter 35 to operate in the vertical direction 

£ Sn^n ♦ d ' r r° n F ° r thG PUrp0S6S ° f thiS examp,e the horizontel direction wi » be described, although 

the operation is essentially the same processing in the vertical direction ^ 

w Jo°[ 3nalySi f 1" the t h ° rizon te' direction, filter controller 31 informs, by way of direction control signal 39. block detec- 
tor 32 temporal & spatial analyser 33 and de-blocking filter 35 to operate in the horizontal directioS 
a ^£2 1 e{ f?°! 32 analyses tne ima 9e for horizontal block distortion. This is carried out in the same way as 
descnbed for the first embodiment of the present invention above, by way of implementing Equation 1 enough «her 
implementations could he used. The signal is then passed to the temporal and spatial analyser 33 which analyses the 

111 JIT T* 00 * ° f deteCtin9 natUra " y ° CCUrrin9 b ' 0CkineSS in an ^ a9e " This is achieved b^arng 
values w,th previous frames so that any motion can be taken into account. For example, it is possible for a naZy 

occurring edge to be a.igned with one of the block boundaries used by the block-coding method used to encode the 
S e ^ CC ° Unt T 9 f ° r ™ tion ' the false detection ° f n ^ra..y occurring blockiness can be significantly reduced 
B ock edge frame store 34 ,s used to store previous image frames to enable the temporal analysis to take place and 
provides a series of previous values to enable this to happen. 

Once horizontal analysis and temporal analysis has taken place, a measure of block distortion can be derived 
A filter control signal 37. representing the detected level of block distortion, is then fed to de-blocking filter' 35 

t H 6Cted ,e ; e ' ° f ^ diSt0rti ° n - d6 - b,0Ckin9 ,ilter 35 applies a filter to the ^ accorSg to 
h L l T Sh ° Wn R9Ure 4 - ' f Severe blockine ss is detected Filter 1 may be used. If. however no blocking 
is detected then Filter 5. which has no filtering effect, is used. oiocwness 

i he filtered output signal 38 can then be fed to storage, display or other means 

Hir J y ™T a * ng T b 'f K detection ' tem P° ral & s P a,ial ana| ysis and de-blocking in both the horizontal and vertical 
direction the overall result is that the input signal is processed as per the first embodiment of the present invention Tte 
advantage of this embodiment, however, is the reduction in the number of modules required to achieve the present 
invention which results in a subsequent reduction in the cost of realising the present invention. 



20 



25 



30 



35 



40 



45 



50 



55 Claims 



1 . A method of filtering an input signal representing a digital block-based coded image to produce an output signal 

the method comprising the steps of: 
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detecting the level of block distortion in the image; and 

filtering the input signal in dependence on the level of block distortion. 

2. The method of claim 1, further comprising detecting the level of block distortion by comparing areas on either side 
5 of boundaries formed between blocks. 

3. The method of claims 1 or 2, further comprising filtering across the boundaries so as to reduce the visibility of block- 
based distortion. 

io 4. The method of claims 1 , 2, or 3, further comprising processing the level of block distortion by comparing it to neigh- 
bouring values both spatially and temporally so as to reduce the false detection of naturally occurring blockiness. 

5. The method of claims 1 , 2. 3 or 4, further comprising analysing the image to determine whether the level of block 
distortion will be masked by spatial detail so as to arrive at a more accurate level of block distortion. 

15 

6. The method of claims 1 , 2, 3, 4 or 5, further comprising filtering the input signal in both the horizontal and vertical 
direction. 

j 

7. Apparatus for filtering an input signal representing a digital block-based coded image to produce an output filtered 
20 signal, the apparatus comprising; 

a detector for detecting a level of block distortion; and 

a filter for filtering the input signal in dependence on the level of block distortion. 

25 8. The apparatus of claim 7, wherein the detector detects a level of block distortion by comparing areas on either side 
of block boundaries formed between blocks. 

9. The apparatus of claims 7 or 8, wherein the filter is applied across the boundaries, so as to reduce the visibility of 
block-based distortion. 

30 

1 0. The apparatus of claims 7, 8 or 9, further comprising a processor for processing the level of block distortion by com- 
paring it to neighbouring values both spatially and temporally so as to reduce the false detection of naturally occur- 
ring blockiness. 

35 11. The apparatus of claims 7, 8, 9 or 1 0, wherein the detector is adapted to determine whether the level of block dis- 
tortion will be masked by spatial detail so as to detect the level of distortion more accurately. 

12. The apparatus of claims 7, 8, 9, 10 or 1 1 , wherein the filter is adapted to lilter smooth across the boundaries in both 
the horizontal and vertical direction. 

40 

13. The apparatus of claim 10, wherein the processor further comprises storing means for storing previous values of 
blockiness for use in the temporal analysis. 
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Fig.1 
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Fig.2. 
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Fig.3. 
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(54) Filtering method to reduce block artifacts in digital video sequences 

(57) The present invention relates generally to the 

domain of digital signal processing and, more precisely, p* ^ 

the methods and apparatus employed in the reduction y " 
of visibility of blockiness in images or video sequences 
encoded using block-based coding techniques such as 
MPEG. 

The present invention detects a level of blockiness 
between blocks in an image and applies a variable filter 
across the block boundaries. The filter parameters are 
chosen depending on the level of blockiness detected. If 
no blockiness is detected then no filtering is performed 
on the image. 

An important aspect of the present invention 
reduces the false detection of naturally occurring block- 
iness in images by analysing the image both spatially 
and temporally. 
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